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UNCLASSIFIED 
T. (WU) PURPOSE: 
CU) Purpose of this report is to review initial STAR GATE 


procedures and to provide recommendations for revisions based on 


recent experience including DESERT STORM tasking. 


(S/NF ) This report examines data development, evaluation and 


assessment procedures for data generated by STAR GATE sources. 


TIT. (WU) SUMMARY: 

A. (S/NF) Lessons learned from recent DESERT STORM tasking have 
led to several immediate changes in STAR GATE procedures as discussed 
in Section IV, Some recommendations for future revisions are also 
identified. 

EB, (S/NF} Evaluations of future STAR GATE reporting will be 
prepared in accordance with the criteria and provisions of DIAM 58-15, 
Subject: The Department of Defense HUMINT Management System, dated 28 


March 19688. 


TV. (U) BACKGROUND: 
A. (S/NF) In December 1990 and January 1991, initial efforts 
were made to provide an effective and viable system to evaluate STAR 


GATE reporting. These procedures are illustrated in Figure 1. The 
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initial proposals resulted in the implementation of procedures that 
attempted to assess both the utility and the value of STAR GATE 
reporting. 

RB. (S/NF) During the course of an off-site conference held on 7 
and 8 March 1991 at Fort Dietrich, senior analysts from DT-SA, DT-3B, 
and DT-8, identified shortfalls in current evaluation procedures and 
prescribed revisions that would allow for the implementation of an 
objective and workable system. The senior analysts believed that 
assessments relating to the accuracy af STAR GATE reporting could 
best be addressed by applying applicable scientific methodologies to 
STAR GATE material. Frocedures already exist and can be implemented 
available project ADF methods. Specific accuracy assessments would be 
determined by intelligence analysts in conjunction with appropriate 
outside scientific review. The same analysts were of the opinion that 
the intelligence value or utility of STAR GATE reporting could best be 
determined by select intelligence analysts directly involved with 
specific task areas. 

C. (U) What follows are the results of a comprehensive group 
effort conducted to identify procedural shortfalls and recommend the 


implementation of the following revisions. 


V. (S/NF/SG/LIMDIS) SHORTFALLS AND REVISIONS: 


~ SHORTFALL: Need to identify action officers with whom DT-3B 


can deal directly to expedite the evaluation process and insure that 


such evaluations are accomplished in the most successful manner. 
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analytical evaluation process. Direct involvement with these (and 
other) scientific analysts should help in better task definition and 


in identifying ways for improving potential utility. 


~ SHORTFALL: Procedures to date appear burdensome and need 
streamlining. 

~ REVISION: Procedures developed by the R&D community should be 
used for assessing the accuracy of project reporting. The 
intelligence value of STAR GATE reporting must be determined by 
intelligence community analysts working from an all source 
intelligence perspective and environment. This should be based on an 


agreed-upon definition of utility and what target data would be 


required to satisfy the utility criteria. 


~ SHORTFALL: Need to assess whether our quantifiable answer 
categories would be expanded to reflect the range of uncertainty when 
one is evaluating a DT-S report. 

~ REVISION: Specificity in the judicious selection of mission 
requirements will go along way towards streamlining the process. 


Target selection must be based on targets against which STAR GATE has 
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a favorable track record. To this extent, limits to STAR GATE data 
(or project type), need to be provided in advance to the tasking 


elements. 


~ SHORTFALL: Need to re-accomplish certain original tasking, 
updating it as necessary to the current situation. 

= REVISTON: Tasking should be levied within specific time 
windows and not be left open-ended. Requirement originators should be 
reminded of their responsibility to update requirements and changes of 


collection emphasis. 


“ SHORTFALL ¢ Senior analysts need ta insure that the 
intelligence evaluations (ia, utility assessments) of project 
reporting are timely, objective, substantive and prepared in 
consonance with established intelligence community procedures. 
~ REVISION: STAR GATE reporting needs to be evaluated in 
consonance with the DIA HUMINT evaluation scheme paraphrased 
substantially as follows: 
RATING VALUE 
Of Major Significance Having Impact on National 
Folicy and Decision-Making 
Of High Value Hast Effort - First Report 
in a Field of Endeavor 


Of Value Supplements ~ Updates - 


Confirms Known Data 
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Of Low Value ; as Foor Quality - Not Timely 

Of No Value Erroneous - Misleading 
There is another consideration. Froper assessment procedures for 
either accuracy or utility require that criteria for evaluation be 
specific in advance. Consequently, it should be appropriate for a 
candidate user (or task originator) to clearly define what type of 
data would have potential utility, and how it would be used if 
provided and how it would be evaluated. This type of data could 
include specific essential elements of information (EEI) and required 
timing. & troublesome problem may be encountered if the specific EEL 
(if posed as a question) provides too much background information for 
the source that could adversely influence session results. To avoid 
such difficulty, it may be necessary to conduct the session in several 
phases, time permitting. This issue is not easy to resolve and 
remains as a future action item. (A basic review of this overall 


issue can be found in DT-S-1010-S, 15 Dec 1990, Evaluation Methods). 


- SHORTFALL: Need to reconsider acceptance of tasking based on 
availability and timeliness of known ground truth. 

- REVISION: The STAR GATE collection management system must be 
selective in its acceptance of requirements for which there is a 
likelihood of learning ground truth within a reasonable amount of 


time. A procedure needs to be established that allows for rapid 


follow-up for task refinement. 
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- SHORTFALL: Multiple source reporting in a combined summary 
format 16 confusing precludes affixing responsibility to the 
reporting. 

~ REVISION: The principle of “one source one report" must be 
strictly adhered ta so that reporting may be attributable ta a 
specific source. Consequently, data from any source will be reported 
separately or in @asily identifiable and separated gactions of an 


averall summary report. 


- SHORTFALL: Current STAR GATE reporting is inconsistent with 
other established HUMINT reporting formats. 

- REVISION: Reports meed to be prepared in accordance with the 
criteria used in the preparation of Intelligence Information Reports 
(IIR‘s). Each report will bear a numerical source designator that 
will protect the identity of the source. An example of the reporting 
format is at Figure &. A Summary of Information (SOI) will be 
prepared by the source. It will reflect a project number, source 
number, session number, and date of report. The SOI will be attached 


as an addendum to the formal report. An example of an SOI is at 


Figure 3. 


~ SHORTFALL: STAR GATE predictions are rarely accurate. 


~ REVISTON: Predictive reporting is indeed the project's 
weakest feature. The results of searches are conducted at a slightly 
higher rate of success. Historically, the project’s track record 
eqummeupay (0) FORN/LIMDIS 
STAR GATE 
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reveals arelatively high rate of success when mission tasking is 
targeted against fixed, static targets located in past or current time 
windows. Consequently, project tasking should be prioritized 


according to estimated success probability. 


~ SHORTFALL: STAR GATE project reporting should not stand alone. 


- REVISION: Senior analysts nead to insure that STAR GATE is 


an integral part of the all-source collection management system, a key 


tip-off player acting in concert with other intelligence disciplines. 


= SHORTFALL: Highlights of singularly outstanding reporting 
should be recorded and made readily retrievable. 

-~ REVISTON: A "Kudo File" will be established to facilitate the 
retrieval of axamples of high quality reporting. Factors that may 
have contributed to these results will be identified whenever 


possible. 
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